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Docket No. 282736US8X 
IN RE APPLICATION OF: Frederic KAPLAN 
SERIAL NO: 10/680,006 
FILED: October 7, 2003 

FOR: ADAPTIVE ARTIFICIAL VISION METHOD AND SYSTEM 

COMMISSIONER FOR PATENTS 
ALEXANDRIA, VIRGINIA 22313 

SIR: 

Transmitted herewith is an amendment in the above-identified application. 

□ No additional fee is required 

□ Small entity status of this application under 37 C.F.R. §1.9 and §1.27 is claimed. 
■ Additional documents filed herewith: Request for Extension of Time (3 Months) 



The Fee has been calculated as shown below: 
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□ Reduction by 50% for filing by Small Entity 


$0.00 


TOTAL 
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□ A check in the amount of $0,00 is attached. 

■ Credit card payment form is attached to cover the fees in the amount of $1,020.00 

■ Please charge any additional Fees for the papers being filed herewith and for which no check or credit card payment is 
enclosed herewith, or credit any overpayment to deposit Account No. 15-0030. A duplicate copy of this sheet is enclosed. 

■ If these papers are not considered timely filed by the Patent and Trademark Office, then a petition is hereby made under 
37 C.F.R. §1.136, and any additional fees required under 37 C.F.R. §1.136 for any necessary extension of time may be 
charged to Deposit Account No. 15-0030. A duplicate copy of this sheet is enclosed. 
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Docket No. 282736US8X 

IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 

IN RE APPLICATION OF: Frederic KAPLAN 

SERIAL NO: 10/680,006 GAU: 2624 

FILED: October 7, 2003 EXAMINER: LIEW, A. K. S. 

FOR: ADAPTIVE ARTIFICIAL VISION METHOD AND SYSTEM 

REQUEST FOR EXTENSION OF TIME 
UNDER 37 C.F.R. 1.136 



COMMISSIONER FOR PATENTS 
ALEXANDRIA, VIRGINIA 22313 

SIR: 

It is hereby requested that a three month extension of time be granted to August 23, 2007 for 
■ filing a response to the Official Action dated: February 23, 2007. 

□ responding to the requirements in the Notice of Allowability dated: 

□ filing the Formal Drawings. The Issue Fee due has been timely filed. 

□ responding to the Notice to File Missing Parts of Application dated: 

□ filing a Notice of Appeal. A timely response to the final rejection, due has been filed. 

□ filing an Appeal Brief. A Notice of Appeal was filed on: 

□ Applicant claims small entity status. See 37 CFR 1 .27. Therefore, the fee amount shown below is reduced by 
one-half. 

The required fee of $ 1 ,020.00 is enclosed herewith by credit card payment and any further charges may be made 
against the Attorney of Record's Deposit Account No. 15-0030 . A duplicate copy of this sheet is enclosed. 

Respectfully Submitted, 

OBLON, SPIVAK, McCLELLAND, 
MAIER & NEUSTADT, P.C. 



Copy Previously Filed and Three-Month 
Extension of Time Paid on August 23, 2007 
Bradley D. Lytle 
Registration No. 40,073 

Customer Number 

22850 

Tel. (703)413-3000 
Fax. (703) 413-2220 
(OSMMN 05/03) 
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IN THE UNITED STATES PATENT & TRADEMARK OFFTPR 



IN RE APPLICATION OF 

FREDERIC KAPLAN 

SERIAL NO: 10/680,006 

FILED: OCTOBER 7, 2003 

FOR: ADAPTIVE ARTIFICIAL VISION 
METHOD AND SYSTEM 



EXAMINER: LIEW, ALEX KOK SOON 



GROUP ART UNIT: 2624 



AMENDMENT UNDER 37 C.F.R. S L1 11 



COMMISSIONER FOR PATENTS 
ALEXANDRIA, VIRGINIA 22313 

SIR: 



Responsive to the Office Action of February 23, 2007, please amend the above- 
identified application as follows: 

Amendments to the Claims are reflected in the listing of claims which begins on 
page 2 of this paper. 

Remarks/Arguments begin on page 9 of this paper. 



Application No. 10/680,006 

Reply to Office Action of February 23, 2007 



IN THE CLAIMS 
Please amend the claims as follows: 

Claim 1 (Original): An adaptive artificial vision method comprising the following 

steps: 

(a) defining successive couples of synchronized timesteps (t.,, t ; t, t+i ;. . .) such that 
the time difference t between two synchronized timesteps (t,, t ; t, t*i ;. . .) of a couple of 
synchronized timesteps is equal to a predetermined time delay x 0 , 

(b) comparing two successive images (I t . b I, ; I t , I t+I ;. . .) at each couple of 
synchronized timesteps (t,, t ; t, t+, ;. . .) spaced by said predetermined time delay x 0 for 
obtaining a delta image A, which is the result of the computation of the distance between each 
pixel of said two successive images (I,.,, I t ; I t , I t+ , ;. . .) in view of characterizing movements 
of objects between said two successive images (I t .,, I t ; I t , I t+1 ;. . .), 

(c) extracting features from said delta image A, for obtaining a potential dynamic 
patch P t which is compared with dynamic patches previously recorded in a first repertory Rj 
which is progressively constructed in real time from an initial void repertory, 

(d) selecting the closest dynamic patch D f in this first repertory R| or if not 
sufficiently close dynamic patch still exists, adding the potential dynamic patch P t to the first 
repertory Rj and therefore obtaining and storing a dynamic patch D; from the comparison of 
two successive images (I t .,, I t ; I t , I t+1 ;. . .) a t each couple of synchronized timesteps (t.,, t ; t, 
t+i ;...), and 

(e) temporally integrating stored dynamic patches Di of the first repertory R, in order 
to detect and store stable sets of active dynamic patches representing a characterization of a 
reoccurring movement or event which is observed. 
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Claim 2 (Original): A method according to claim 1, wherein when stable sets of 
active dynamic patches representing a characterization of a reoccurring movement have been 
detected, the center of the movement is identified and static patches which are at a 
predetermined distance d from the movement center and are obtained by a process of static 
pattern recognition are analyzed to constitute at a given timestep a set of active static patches 
Si which are stored in a second repertory R s . 

Claim 3 (Original): A method according to claim 2, wherein stored patches Si of the 
second repertory R s are spatially integrated in order to detect and store stable sets of active 
static patches representing a characterization of an object which is recurrently involved in 
observed known reoccurring movements. 

Claim 4 (Currently Amended): A method according to claim 2 or claim 3 , wherein 
the process of static pattern recognition and production of static patches is initiated after 
stable sets of active dynamic patches representing a characterization of a reoccurring 
movement have been detected. 

Claim 5 (Currently Amended): A method according to claim 2, An adaptive artificial 
vision method comprising the following step s: 

(a) defining successive cou ples of synchronized timestep s (t, K t : t Ut ;. . .) such that 
the time difference t between two synchronized timesteps (t u t : t t+i . .) of a couple of 
synchronized timesteps is equal to a predetermined time delay x n. 

(b) comparing two successive images (l uu I : k L+i ;. . .) at each couple of 
synchronized timest eps (t^t ; t Ui . .) spaced bv said predetermined time delay t , for 
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obtaining a delta image A, whic h is the result of the computation of the distance between each 
pixel of said two successive i mages (I t . N I t ; I t , I t+ , ;, . ,) j n view of characterizing movements 
of objects between said two successive images (1 <_k L : T,. T^^J^ 

(c) extracting features fr om said delta image A for obtaining a potential dynamic 
patchPt which is compared with dynamic patches previously recorded in a first repertory Rh 
which is progressiv ely constructed in real time from an initial void repertory. 

(d) selecting the closest dynamic patch D; in this first repertory Rh or if not 
sufficiently close dynamic patch stil l exists, adding the potential dynamic p atch P, to the fW 
repertory Rh and therefore obtaining and storin g a dynamic patch D ; from the comp arison of 
two successive images (I t . u ,I t ; I,, I t+I ;...) at each couple of synchronized timestep s (t^ t ; t 
t+i and 

(e) tempor ally integrating stored dynamic patches Pi of the first rep ertory R d in order 
to detect and store stable sets of activ e dynamic patches rep rese nting a characterization of a 
reoccurri ng movement or event which is observed. 

wherein when stable sets of active dyna m ic patches representing a characterization of 
a reoccurring movement have been de t ected, the center of the movement is identified and 
static patches which are at a predetermin e d distance d from the movement center and are 
obtained by a process of static pattern r ecognition are analyzed to constitute at a g iven 
timestep a set of active static patches S; which are stored in a second repertory R ,. and 
wherein the process of static pattern recognition and production of static patches is initiated at 
the same time as the process of dynamic movement recognition and production of dynamic 
patches and when stable sets of active dynamic patches representing a characterization of a 
reoccuring movement have been detected, the process of static pattern recognition is 
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continued exclusively with static patches which are located in a restricted area of the image 
which is centered on said identified movement center. 

Claim 6 (Currently Amended): A method according to claim 1, wherein during the 
computation of the distance between each pixel of two successive images (I t . 1? I t ), a filter 
function f th is used to keep only the most significant differences and therefore obtain a delta 
image A t such that 

Claim 7 (Original): A method according to claim 6, wherein the filter function f th is a 
threshold function. 

Claim 8 (Currently Amended): A method according to any one of claims 1 to 7 claim 
I, wherein the step of extracting features from the delta image A t comprises computing a 
gaussian color model of the distribution for each color component. 

Claim 9 (Currently Amended): A method according to any on e of claim s 1 to 7 claim 
I, wherein the step of extracting features from the delta image A t comprises using histograms 
to model the distribution for color components, shape or texture. 

Claim 10 (Currently Amended): A method according to any on e of claims 2 to 5 
claim2, wherein static patches are obtained on the basis of salient points (x,y) in an image I t 
provided at a synchronized timestep t when a salient point (x,y) is detected, a region R x y 



5 



Application No. 10/680,006 

Reply to Office Action of February 23, 2007 

corresponding to the surrounding pixels is defined and features are extracted from this region 
R Xjy to define a potential static patch S x , y . 

Claim 1 1 (Original): A method according to claim 10, wherein the extraction of 
features from the region R xy comprises measuring the color change of a pixel compared to its 
neighbors and computing a color model of the color distribution in the region R xy . 

Claim 12 (Currently Amended): A method according to anyone of claims 1 to 11 
claimJL wherein successive steps of synchronized timesteps (L u t ; T, T +1 ;. . .) are separated 
by a period of time T which is equal to n times the predetermined time delay x 0 , where n is an 
integer which is positive or equal to zero. 

Claim 13 (Original): A method according to claim 12, wherein successive couples of 
synchronized timesteps (t u t ; t, Ui ;. . .) are contiguous without any time interruption 
between two successive couples of synchronized timesteps (t. l5 1 ; t, Ui). 

Claim 14 (Currently Amended): A method according to any one of claims 1 to 13 
claiml, wherein it further comprises the step of detecting transitions between stable sets of 
active dynamic patches representing a characterization of reoccurring movements and of 
constructing transition graphs for predicting complex events comprising a sequence of 
identified movements. 

Claim 15 (Currently Amended): An adaptive artificial vision system comprising: 
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- a clock [[(101)]] for defining successive couples of synchronized timesteps (t_i, t ; t, 
t+i ;. . .) such that the time difference x between two synchronized timesteps (L\, t ; t, t+j ;. . .) 
of a couple of synchronized timesteps is equal to a predetermined time delay x 0 , 

- inputting means [[(102)]] for inputting images (I,.,, I t ; I„ I t+1 ;. . .) provided by a 
camera [[(103)]] at said synchronized timesteps (ti, t ; t, t+i ;...), 

- first comparator means [[(104)]] for comparing two successive images (I t . b I t ; I t , 
;. . .) inputted at each couple of synchronized timesteps (t.,, t ; t, t+, ;...) spaced by said 

predetermined time delay x 0 for obtaining a delta image A, which is the result of the 
computation of the distance between each pixel of said two successive images (I M , I t ; I t , 

it+i ;•••)> 

- first memory means (M d ) for storing dynamic patches Di representing elementary 
visual parts for describing characterized movements of objects, 

- feature extraction means [[(105)]] for extracting features from said delta image A, 
and producing a potential dynamic patch P,, 

- second comparator means [[(106)]] for comparing a potential dynamic patch P t 
which is compared with dynamic patches previously recorded in said first memory means 
(M d ), 

- selection means [[(107)]] for selecting the closest dynamic patches D s in the first 
memory means (M d ) or if no sufficiently close dynamic patch still exists, for recording the 
potential dynamic patch P t into the first memory means so that a dynamic patch Dj is stored in 
the first memory means for each comparison of two successive images (I M , I, ; I t , I t+1 ;. . .) at 
each couple of synchronized timesteps (ti, t ; t, t +1 ;. . .), 

- first temporal integrations means [[(108)]] comprising computing means [[(108 A)]] 
for computing during a time T F1 corresponding to a predetermined number Nl of couples of 
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synchronized timesteps the frequency of each dynamic patch D; stored in the first memory 
means and threshold means [[(108B)]] for defining a set of active dynamic patches 
comprising dynamic patches D; whose frequency is higher than a predetermined threshold, 
and, 

- second temporal integration means [[(107)]] comprising computing means 
[[(109A)]] for computing during a time T F2 corresponding to a predetermined number N2 of 
couples of synchronized timesteps the frequency of each set of defined active dynamic 
patches and threshold means [[(109B)]] for defining a stable set of dynamic patches 
corresponding to a reoccuring movement for each set of active dynamic patches whose 
frequency is higher than a predetermined threshold. 

Claim 16 (Currently Amended): A system according to claim 15, wherein it further 
comprises means [[(1 10)]] for identifying the center of a reoccuring movement represented 
by a stable set of active dynamic patches and means [[(1 1 1)]] for triggering static pattern 
recognition [[(1 12)]] for analyzing static patches which are at a predetermined distance d 
from said center of a reoccuring movement. 
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REMARKS/ARGUMENTS 
Favorable reconsideration of this application as presently amended and in light of the 
following discussion is respectfully requested. Claims 1-16 are pending, Claims 4-6, 7-10, 
12, 14, 15 and 16 having been amended by way of the present amendment. 

In the outstanding Office Action Claims 6 and 7 were objected to as containing 
informalities; Claim 5 was objected to as being dependent upon a rejected base claim, but 
otherwise contains allowable subject matter; Claims 1 and 12-14 were rejected as being 
unpatentable over Cohen (U.S. Patent No. 6,681,031) in view of Dobashi (U.S. Patent 
Publication No. 2002/0126880); Claims 2-4, 10, 15 and 16 were rejected as being 
unpatentable over Cohen in view of Dobashi and in further view of Bongiovanni (U.S. 
Publication No. 2005/0041 102); Claims 6 and 7 were rejected as being unpatentable over 
Cohen in view of Dobashi and in further view of Banh (U.S. Patent No. 5,150,426); Claims 8 
and 9 were rejected as being unpatentable over Cohen in view of Dobashi and in further view 
of Kim (U.S. Patent No. 6,999,604); and Claim 1 1 was rejected as being unpatentable over 
Cohen in view of Dobashi . Bongiovanni and in further view of Kim . 

In reply, Applicants first would to thank the Examiner for identifying allowable 
subject matter. Claim 5 has therefore been written in independent form and is believed to be 
in condition for formal allowance. The informality identified in Claim 6 has been corrected 
by way of the present amendment. Therefore it is believed that the objections to Claims 6 
and 7 has been overcome. 

Claim 1 is directed to an adaptive artificial vision method that, among other things, 
includes steps (c) extracting features from said delta image A, for obtaining a potential 
dynamic patch P t which is compared with dynamic patches previously recorded in a first 
repertory Rj which is progressively constructed in real time from an initial void repertory, 
and (d) selecting the closest dynamic patch D ; in this first repertory R, or if not sufficiently 
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close dynamic patch still exists, adding the potential dynamic patch P, to the first repertory R d 
and therefore obtaining and storing a dynamic patch D ; from the comparison of two 
successive images (I„,, I, ; I t , I t+1 ;. . .) at each couple of synchronized timesteps (t,, t ; t, 
Ui ;...)• 

Thus, a repertory is initially void and a first repertory is progressively constructed 
in real time. In contrast, in Cohen a predetermined number of gestures are identified and the 
different types of gestures are divided in different lines. Cohen provides an initial repertory 
which is not void and is progressively constructed in real time. 

According to Cohen, a reference database includes a limited number of patches 
corresponding to a limited number of stereotype gestures (see Abstract "Feature position 
measure is used in conjunction with a bank of predictor bins seeded with the gesture 
parameters, and the system determines which bin best fits the observed motion"). 

As acknowledged in the outstanding Office Action, according to Cohen , in the 
identification process, a closest gesture in the reference bins will be selected and if there is 
none, then no gesture is identified (column 21, lines 54-64). Cohen thus discloses a non- 
flexible system which needs a preliminary identification of specific gestures and cannot be 
further expanded during a recognition process. This is a very substantial difference with 
respect to the subject matter defined by Claim 1 . 

Dobashi cannot reasonably be combined with Cohen because Dobashi relates to a face 
image recognition apparatus and a face image can hardly be defined as a recurring movement 
or element. Moreover, contrary to step c) in Claim 1, Dobashi requires a preset registration 
procedure (see page 6 [0090]) and cannot start with a void register. Furthermore, when a new 
reference feature amount is registered into the registration information holding section 
([0086]), such new reference feature is considered as being a second class type of information 
([0090]) "the old registration information is a feature amount registered by use of a preset 
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registration procedure and is highly worth holding and the new registration information is a 
feature amount extracted in the course of the recognition process and is less worth holding"). 
Accordingly, there is no incentive for a person of ordinary skill in the art to combine Dobashi 
and Cohen. Nevertheless, whether taken individually or in combination, neither Cohen nor 
Dobashi teach or suggest the claimed method since neither Dobashi or Cohen starts from a 
void register. Consequently it is respectfully submitted that Claim 1 patentably defines over 
Cohen in view of Dobashi . 

Claims 2-4 and 6-14, as amended, all depend from Claim 1. Thus it is believed that 
these claims also patentably define over the asserted prior art. 

Claim 1 5 relates to a corresponding system and therefore is of a different statutory 
class. But nevertheless, Claim 15 requires "selection means for selecting the closest dynamic 
patch Dj in the first memory means or if no sufficiently close dynamic natch still exists, for 
recording the potential dyna mic p atch P, into the first memory means ." As discussed above 
with regard to Claim 1, Cohen describes a non-flexible system which needs a preliminary 
identification of specific gestures and cannot be further expanded during a recognition 
process. Dobashi does not cure this deficiency because it does not disclose particular 
memory means specifically adapted for storing dynamic patches representing elementary 
visual parts for describing characterized movements of objects. Thus it is respectfully 
submitted that no combination of Cohen in view of Dobashi teaches or suggests all the 
elements of Claim 15. Because Claim 16 depends from Claim 15 it is respectfully submitted 
that Claim 16 also patentably defines over the asserted prior art. 

Consequently, in view of the present amendment and in light of the foregoing 
comments, it is respectfully submitted that the invention defined by Claims 1-16, as amended, 
is patentably distinguishing over the prior art. The present application is therefore believed to 
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be in condition for formal allowance and an early and favorable reconsideration of this 
rejection is therefore requested. 

Respectfully submitted, 

OBLON, SPIVAK, McCLELLAND, 
MAIER & NEUSTADT, P.C. 



Customer Number 

22850 

Tel: (703)413-3000 
Fax: (703)413 -2220 
(OSMMN 08/07) 




Bradley!). Lytle 



Attorney of Record 
Registration No. 40,073 
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